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Abstract 

In order to explain the empirical evidence that the dynamics of hu- 
man activity may not be well modeled by Poisson processes, a model 
based on queuing processes were built in the literature (Barabasi, 
2005). The main assumption behind that model is that people ex- 
ecute their tasks based on a protocol that execute firstly the high 
priority item. In this context, the purpose of this letter is to ana- 
lyze the validity of that hypothesis assuming that people are rational 
agents that make their decisions in order minimize the cost of keeping 
non-executed tasks on the list. Therefore, we build and solve analyti- 
cally a dynamic programming model with two priority types of tasks 
and show that the validity of this hypothesis depends strongly on the 
structure of the instantaneous costs that a person has to face if a 
given task is kept on the list for more than one step. Moreover, one 
interesting finding is that in one of the situations the protocol used to 
execute the tasks generates complex one dimensional dynamics. 

1 Introduction 

Empirical evidence has shown that the dynamics of inter-event times driven 
by human actions may not be random and not well approximated by Poisson 
processes (Paxson and Floyd, 1996; Masoliver et al., 2003; Scalas et al., 2006). 
Based on this, Barabasi (Barabasi, 2005) developed a very interesting model 
of human activity where the distribution of the inter-event time is a con- 
sequence of a decision queue processes. He considers that among the most 
relevant protocols for driving human dynamics, e.g. first-in-first-out protocol, 
random protocol and a protocol based on the execution of the high priority 
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item, this later protocol seems to be the most important. In this protocol, 
while high priority tasks are executed as soon as they are added to the list, low 
priority tasks wait for a long time until all high priority tasks are executed, 
i.e., the instants of execution of low priority tasks are separated by long times 
of inactivity. Using this assumption, it is numerically (Barabasi, 2005) (an- 
alytically (Vazquez, 2005)) shown that the distribution of inter-event times 
follows a power law. 

Two interesting contributions were introduced by (Grinstein and Linsker, 
2006). First the authors map the variable length priority model considered 
above onto a model of biased diffusion deriving asymptotic distributions for 
the inter-event times. Second, in order to investigate the arising of power 
laws in more general situations, they generalize the fixed length model queue 
to contain tasks with a priority label and with a class label where there is 
always an active class and an inactive class. If the highest priority task of 
the inactive class exceeds that of the active class by at least a fixed switching 
cost, the inactive class becomes active and the active class becomes inactive. 

An interesting discussion is considered in (Kentsis, 2005; Barabasi and 
Oliveira, 2006) where it is argued that other mechanisms contribute for the 
distributions of waiting times such as deadlines, time dependence of priorities 
and the social context of the problem. In line with this debate, (Blanchard 
and Hongler, 2007) relaxes the assumption that the priorities of tasks do not 
change over time and studies queueing systems where deadlines are assigned 
to the incoming tasks and the urgency to attend a task increases with time 
showing that only in the former model fat tails arise naturally as consequence 
of the scheduling rule. 

In this letter, we investigate the assumption that people execute tasks 
on a protocol that execute firstly the high priority item. In particular, we 
suppose that people assign priorities to the tasks on their lists in order to 
minimize some cost index, i.e., a cost associated to the fact of not processing 
a given collection of tasks in a given time step. Therefore, based on this 
assumption and inspired on (Barabasi, 2005; Vazquez, 2005; Grinstein and 
Linsker, 2006), we have built a discounted stochastic dynamic programming 
model with two types of tasks (low and high priority tasks) and a cost per 
stage for keeping a number of low and high priority tasks without processing. 

This is not the first time that a kind of optimization principle is used to 
understand the structure and dynamics of complex systems. In (Rodriguez- 
Iturbe et al., 1992; Gajueiro, 2005), for instance, it is shown that complex 
networks may arise from optimization principles. 

It is also important to stress that although there is a large literature 
dealing with control of queue discipline (Crabill ct al., 1977; Kitacv and 
Rykov, 1995; Sennott, 1999) which this work is related, the model presented 
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in this letter is neither an extension nor a particular case of any of these 
results. 

We have found that the type of protocol used to execute tasks is strongly 
dependent on the kind of instantaneous cost of keeping a task in the queue 
for an additional stage. When linear costs are used the protocol of executing 
preferentially the high priority costs always is the best solution. However, this 
does not happen when quadratic costs are considered. In this case different 
types of protocol are considered. Furthermore, depending on the parameters 
of the system, the protocol considered generates complex one dimensional 
dynamics. 

2 Setup of the problem 

We consider that there are two queues waiting for a service on a single server. 
Let g{xL, xh) be the current cost of having state {xl, xh) which is the state 
of the system, (xh) is the number of tasks in the first (second) queue. 
We say that the first queue is a low priority queue (or the second queue 

is a high priority queue) if ^^^^^f^^\xL=XH < ^^%xh"^ l^L—rH- assume 
that this is the case. The dynamics of these queues are modeled as follows: 
At each discrete time step with probability Xp a new task arrives in the 
queue formed by high priority tasks and with probability A(l — p) a new 
task arrives in the queue formed by low priority tasks. Within each of the 
queues the tasks arc executed on a First-ln, First-Out basis. With proba- 
bility hu{xl,xh) the first task of the high priority queue is executed and 
with probability //(I — u{xl,xh)) the first task of the low priority queue 
is executed. We assume here that u{xl,xh) is a state dependent control 
variable that the agent will choose in order to minimize the total cost func- 
tion Ju{xl,Xh) = E'^i^xJY.Zi(^^9{{xL{t),XHit)))], where ^"^,^^[-] is the 
expected value conditioned to the current state {xl,xh) and the state con- 
trol variable u and a is the discount factor. 

Due to the principle of optimality (Bellman, 1957; Bertsekas, 2001) and 
the Banach fixed point theorem, if the minimum cost function J{xl,Xh) = 
mmu(xL,XH)&[o,i] Ju{.Xl,Xh) exists, it must be given by the unique solution of 
the Bellman equation, that may be written as 

J{xl,xh) ^ F{xl,xh) + min u{xl,xh)G{xl,xh) (1) 

u{xL,XH)£lO,i\ 
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where 



= g{xL,XH) + \p{l- IJ')[aJ{xL,XH + 1)] 

+ X{1 - p){l - fi)[aJ{xL + l,XH)] 

+ (1 - X)fi[aJ{xL - 1,xh)] 

+ pXp[aJ{xL - 1,xh + 1)] 

+ {1 - p)Xp[aJ{xL,XH)] 

+ {1 - X){1 - p)[aJ{xL,XH)] 



(2) 



and 



+ 



+ 



G{xl,xh) = 

(1 - X)p[a{J{xL,XH - 1) - JixL - 1,xh))] 

pXp[a{J{xL,XH) - J{xl -1,xh + 1))] 

(1 - p)Xp[a{J{xL + 1,xh - 1) - J{xl,xh))] 



(3) 



Since the optimization problem ([T]) is a hnear programming problem, the 
optimal control u{xl, xh) in each state {xl, xh) will depend exphcitly on the 
signal of G{xl, xh)- If G{xl, xh) > 0, then u{xl, xh) = 0. If G{xl, xh) < 0, 
then u{xl,xh) = 1- Finally, if G{xl,xh) = 0, u{xl,xh) is a mixed strategy 
that may present any value in the interval [0,1]. It is quite intuitive this 
result. Indeed, one may note that the terms in square brackets defined in 
G{xl,xh), equation (I3l), comprises the variations in the cost function due 
to changes in the states of the queue related to the execution of one of the 
tasks. 

Since the properties of the solution of J{xl, xh) of the Bellman equation 
([T]) are strongly dependent on choice of the cost per stage g{xL,XH), in the 
next sections, two different choices for g{xL,XH) are investigated. 

3 Linear costs 

In this section, we assume that g{xL, xh) = hixi + hnxn, for < Hl < hn, 
i.e., the current cost of having one additional high priority task in the queue 
is larger than having one additional low priority task in the queue. 

Since the space of polynomials of degree 1 with sup-norm is a Banach 
space, one can show inductively, making recursive iterations of the dynamic 
programming mapping, that J{xl,xh) is also linear. Therefore, for xl > 
and Xh > 0, guessing this form, one may easily solve the Bellman equation 
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(PP) and show that the cost function is given by 

J{xl,xh) = c + clXl + chXh (4) 

Furthermore, 

G{xl, xh) = fi- {hi - hn) (5) 

1 — a 

is always negative implying that u{xl, xh) = u = 1 for every state (xl, xh)- 
Therefore, if linear costs are considered, the protocol to be considered is the 
one based on the execution of the high priority task whenever there is at 
least one item in this queue, i.e., xh > 0. This kind of protocol was very 
well studied in (Barabasi, 2005; Vazquez, 2005; Grinstein and Linsker, 2006) 
where analytic results for the emerging of power laws may be found. In the 
next section, a much wealthier situation happens where the optimal policy 
is not only limited to execute the high priority item in the queue, but the 
optimal policy is state-dependent. 



4 Quadratic costs 

Now, we assume that q^xl^xh) = hLx1 + hnx'j^, for < hi < hn- Following 
the same reasoning already presented before for the linear cost case, one may 
conclude a quadratic form for the cost function. 

Solving the Bellman equation, one may show that the solution of the 
problem depends explicitly on the signal of the function G{xl,xh), defined 
in ([3]), in the state {xl,xh)- In fact, three different regions will arise. We 
will call region A the domain of {xl,xh) where G{xl,xh) > 0, region B 
the domain of {xl,xh) where G{xl,xh) = and region C the domain of 
{xl,xh) where G{xl,xh) < 0. 

We have found that the minimum cost function is given by 

{J^{xl,xh) if {xl,xh) e A 
J^{xl,xh) if {xl,xh) e B (6) 
J'^{xl,xh) if (xljXh) e C 

and the optimal control is given by 

( Oif{xL,XH)eA 
u{xl, Xh) = I ue [0, 1] if {xl, xh) G B (7) 
[ 1 if {xl,xh) e C 

^The constants are given by cl = j^, ch — ji^ and c — (i^^yi (^(1 ^ + {^P — 
n)hH)- 
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where for i = A, B,C 



(8) 



G\xl, xh) = /u[cli - Chi + ^^{^12X1 - ch2Xh) 

+ CL2{2K^-p)-l)+CH2{l-2\p)] 

(9) 

and G{xl,xh) = G'{xl,xh), if {xl,xh) G 

Furthermore, the parameter 6 G [6, 6], defines the set of points of 3fJ^ such 
that G{xl, {hL/hH)xL + 5) = 1. 
Indeed, 

^ (J^(<5) ^ J^, ^G^) (10) 

and 

5^5^ (J^(5) ^ jc^, ^G^) (11) 

In order to understand the intuition behind this solution, one is invited to 
consider a particular case where hi/hu ~ 1, A = /i = 1 and p = 1/2. In this 
case, the region B is defined by the set of points where \xh — xl\ < a/{l—a). 
Therefore, the set of points where the decision maker can use any strategy 
depends strictly on the discount factor. If the discount factor is large, the 
decision maker may keep queues with a large difference between their sizes. 
On the other hand, if the discount factor is small this situation is not accepted 
as a solution anymore. 

Differently from the linear costs case, several types of protocol are possi- 
ble. Region C considers a protocol based on the execution of the high priority 

2The constants are given by cl2 ^ ch2 ^ 4, - 2a/.,(-^+A(i-p)) ^ ^ 

^,,^{(^[l-2A(l-p)]+ (f^[2a(A-A^) + (2Ap-l + 25)(l--a)], c£i = ^^^f^^, 

4i = if^' 4i('^) = (7rrii;7)i(lSl) [l-2Ap-2<5]+ (^[2(A-^a)-(2Ap+l)(l-a)]}, 

cgi = ^"Yi'-T)^''^ = Ti^{\i^ - + (1 - P)^)) + M^' + - A^A) + 2pA(A.(l + 
a) + Xa{p - 2)) - 2nX]hL + Xp[l -a + 2Xpa]h„, = {[2Aq(1 - a){-p^X + 

2Ap - 1 - A - p)]hl + [6{2Xpa'^ + 4Aa - 4Apa + 1 - 2A - 2a + - 2Xa'^ + 2Xp) + 2X{-pa + 
2Xp^a^ + 2Apa - 2Xp^a - o?p - 2Xo? p + Xo?) + 2/i(a - o? + po?)\hLhn + \b{-^Xpa + 
26-c? ^ 28c? + 2Xpa^ - 1 - 4a(5 + 2a + 2Ap) + 2Xp(Xpc? - Xpa - a^ + a)]h%} and c'^ = 
jj^{[Xil'^a)il-p) + 2X^a{l~pf]hL + [il~a)ip + Xp) + 2p^a + 2Xp{Xpa-pa-p]hH}. 

■^Thc constants S_ and 5 are respectively given by 6— — Ap] + ^[—-^-^ + 

A(l -p)- ap]} and 6 = - Xp + ap] + ^[-^ +\l - p)]}. 
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Xh 



Xh = {hL/hH)xL + 5 

Xh = {hL/hH)xL + 5 




Figure 1: The regions A, B and C in the plane xl — xh- 

task. Region A considers a protocol based on the execution of the low pri- 
ority task. It occurs in order to avoid that the size of the queue of the low 
priority tasks do not increase too much. "Too much" here is measured by the 
ratio hi/fiH- Region B does not determine a protocol. It can be a random 
protocol (mixed strategy) or simply a protocol such the one considered in 
region C or region A. Figure [T] shows the geometry of these regions in the 
plane xl — xh- 

It is not difficult to show that the expected value of the state obeys the 
following dynamics 



Et[x{t + l)] = Et 



' XLit + l) ' 




' XLit) ' 


_ XH{t+l) _ 




_ XH{t) _ 



+ 



A(l - p) - Ai(l - u{xL{t),XH{t))) 

Xp - fiu{xL(t),XH(t)) 



(12) 



which has infinite fixed points if and only if A = p and u{xl, xh) = u = p. 

We will analyze only the most interesting situation which is the fixed- 
length-queue, i.e., A = /i. Therefore, assuming that X = p., = p + e and 
e > 0, then the expected value of the system is governed by Et[x(t + 1)] = 
x(t) + Xee if it is in region B and by Et[x(t + 1)] = x{t) — Ape if it is in region 
A (if the state is in region C, the expected state will certainly come to region 
B and not come back to this region), where e = (1,-1)'. Therefore, the 
dynamics takes place in the line passing by a;(0) and following the direction 
e. Thus, if the expected state is in region B it goes into the direction of 
region A and viceversa. This dynamics is equivalent to the one dimensional 
system 



yit + l) = 

defined on the interval {—t^,V 



y{t) + t+ if y{t) < 
y{t) - t- if yit) > 

where = Xe and t~ 



(13) 



Ap 
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Figure 2: The evolution of y(t) for yo = —0.2, A = 0.5, p = 0.5 and e = 0.3. 



The dynamics of this system is plotted in figure [2]for the case of t" = 0.25 
and = 0.15. The dynamics defined in ffT^ is topologically conjugate 
with the translation in the circle (DeMello and VanStrien, 1993). Therefore, 
if t~^/t~ = p/q, where p/g is a irreducible ratio representation of rational 
number, then this system follows a limit cycle with period p + q. Otherwise, 
the ci;-limit of any point in the interval is a dense subset of it. Therefore, 
we can conclude that the stochastic process that defines the length of each 
queue is not stationary. Moreover, the dynamics of the expected value of 
the length of the queue exhibits a complex behavior: infinitely many cycles 
or a a;-limit set being a dense subset in the interval. The intuition behind 
that complex dynamics is quite reasonable. In the region close to the frontier 
Xff = {hL/hH)xL + 5 that separate A and B (see figure [T]), we can observe 
the following: if the expected state is in A, its dynamics moves toward region 
B, since the priority is of L . Once the expected state is in 5, the dynamics 
takes it back to the region A, since in this case in average the priority is of H 
(due to the condition = p + e and e > 0). Because the frequency of tasks 
arriving is equal to that of attending them, a cyclical or complex dynamics 
emerges close to the referred frontier. Figure [2] shows the case where this 
system is a limit cycle. A similar situation involving regions B and C arises 
in the case of e > and p = + e. In these situations, the protocol is ruled 
by the protocols considered in regions A and B in the former case and by 
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the protocols considered in regions B and C in the later case. 

For \ ^ jji, either the expected value goes to infinite, converges to 0, 
to axis xl = or to axis xh — 0, following different routes. Furthermore, 
different kinds of protocols are possible. 

5 Final Remarks 

In the human dynamics of the tasks execution decisions the priority of one 
task is not always defined as being the most important current task. Actu- 
ally, the dynamics of the work executions depends on the cumulated tasks of 
short run priorities, the importance of each kind of task and the intertem- 
poral discount factor. In this letter, we provide a stochastic dynamic pro- 
gramming model containing all those elements and analyze the dynamics 
of the execution of tasks, shedding new light to the discursion considered 
in (Kentsis, 2005; Barabasi and Oliveira, 2006). In this setting, we have 
found that the dynamics of the expected state of the system may be com- 
plex, exhibiting cycles of any order or with limit set being a dense subset 
of the interval depending on the parameter values of the model. This is a 
contribution to a better understanding of how human dynamics may evolve 
in this type of problem. Finally, it is worth noting that complex dynamics in 
the solution of dynamic programming problems are usually obtained for low 
discount factors (Montrucchio and Sorger, 1996). However, in our quadratic 
case, complex dynamics arises for discount factors of any size. 
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